Automated Decision 
Systems Task Force 


Checklist for Determining whether a Tool or System 
is an ADS/Agency ADS as defined by Local Law 49 (2018) 


Question 1: Is the tool or program an "automated decision system" (ADS)? YES □ NO □ 

Legal definition^ 

Local Law 49 of 
2018, § 1(a). 

Automated decision svstem (ADS): Computerized implementations of algorithms, 
including those derived from machine learning or other data processing or artificial 
intelligence techniaues, which are used to assist in making decisions. 


In determining whether a tool or program constitutes an ADS as defined by Local Law 49 of 
2018, answer "yes" or "no" to the following questions. If answering "yes" to questions 
(l)(a) through (l)(c), check "yes" for Question #1 above, and proceed to Question #2. 

(l)(a) 

Is it an algorithm? YES □ NO □ 

• A set of formal or informal rules, processes, or instructions for carrying out a 
specified operation or solving a problem. 

For informational purposes only, indicate whether the algorithm was derived from any of the 
following:^ 

• Machine learning □ 

The algorithm derives from a process by which a software component or other 
computerized implementation identifies patterns or regularities from input data. Key 
types ol machine learning may include, but are not limited to (in varying degrees of human 
interaction): 

0 Supervised learning (where an algorithm uses data and human feedback to learn 
relationships of given input to output); 

0 Unsupervised learning (where an algorithm explores input data without being given 
explicit output variables); and 

o Reinforcement learning (where an algorithm learns to perform a task by trying to 
maximize rewards it receives for its actions). 

• Data processing □ 

The algorithm derives from an operation performed on data, such as collection, recording, 
organization, storage, adaption or alteration, retrieval, consultation, use, disclosure, 
transmission, dissemination or otherwise making available, alignment or combination, 
blocking, erasure, or destruction. 

• Artificial intelligence (Al) techniques □ 

The algorithm derives from methods that simulate human thought or processes, language, 
and intelligence. Al techniques may include, but are not limited to: 

0 Natural language processing (software or a computer understanding, interpreting, 
and manipulating human language); 

0 Neural networks (using algorithms modeled after the human brain to recognize 
patterns); 

0 Regression analysis (finding relationships between two or more variables of interest 
by examining influence of one or more independent variables on a dependent 
variable); or 

o Markov decision process (decision model for situations in which outcomes are in 
part random and in part under control of a decision maker). 

• Other □ 


^ For purposes of ADS Task Force's work, clarification of relevant key terms that are included in Local Law 49 of 2018, but not defined 
the Law (i.e., underlined terms), is provided below. 

^ Note, the language used in (l)(a) to describe the below terms in this subsection is provided solely as guidance to help clarify their 
plain language meaning for informational purposes, recognizing that in the emerging ADS field, there may be alternate and/or more 
technical descriptions. 
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(l)(b) 

Is it a computerized implementation of an algorithm? YES □ NO □ 

• An algorithm that takes the form of, or is it otherwise partially or entirely operated or 
controlled by, software or a computer. 

o 

(l)(c) 

Is the algorithm used to make or assist in making a decision? YES □ NO □ 

• A determination made by a City official/employee which may impact the actions, 
policies, procedures, services, programs, employment matters, contracting practices, 
rulemaking, or budgeting/allocation of resources, that is implemented by the City or 
an individual agency. 

• Decisions may be related to, but are not limited to: 

o Granting or denying a permission or right; 
o Assigning status or eligibility; 

o Assigning the type, level, or amount of assistance, benefit or service an 

individual, household, or population does/does not receive (e.g. housing, cash 
assistance, medical assistance, etc.); 
o Allocating resources to City programs or services; 

o Furthering civil and criminal law enforcement activities (e.g., investigation, 
surveillance, arrest, pre-trial detention, sentencing or corrections) 
o Responding to public safety/public health concerns (e.g., through health 
promotion or disease prevention techniques); 
o Assigning value to assets (e.g. property valuation); or 
o Auditing programs or entities. 


If any of the above questions is answered "no", then the system or tool under review is not 
considered an "automated decision system (ADS)" as defined by Local Law 49 (2018). 
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Question #2: Does the ADS constitute an "Agency ADS"? YES □ NO □ 

Legal definition 

Local Law 49 of 
2018, § 1(a). 

Automated decision system, agency (Agency ADS): An ADS used by an agency to 
make or assist in making decisions^ concerning rules, policies, or actions 
implemented that impact the public. 


In determininp whether a tool or propram constitutes on Apencv ADS os defined bv Locoi 

Low 49 of 2018, onswer "yes" or "no" to the foiiowing puestions. Ifonswering "yes" to 
puestions (2)(o) ond (2)(b), check "yes" for Question #2 obove. 

(2)(a) 

Is the ADS used by an agency to make or assist in making decisions concerning ruies, 
poiicies, or octions implemented by the City or a City agency? YES □ NO □ 

• Ruies refer to regulations promulgated by a City agency, either pursuant to 
the City Administrative Procedure Act (CAPA), or by a federal, state or other 
agency where the City agency is tasked with implementation. 

• Poiicies refer to principles, procedures, or protocols adopted by an agency to 
guide the agency's practices. 

• Actions refer to affirmative acts/steps the agency takes, as well as inaction by 
an agency. 

(2)(b) 

Do/will such decisions impoctthepubiic? YES □ NO □ 

• The term "impact" is understood as having a moterioi impoct on an individual 
member of the public or a population, in connection with a City agency's use 
of an ADS. In determining whether an impact is material, consider, among 
other factors, whether the decision: 

o Pertains to a procedural or substantive right granted under law, 
regulation, or by City policy; 

o Relates to an individual's or population's protected status; 
o Relates to an individual's or population's eligibility for, or receipt or 
denial of a City or agency program, service, or benefit; 
o Affects whether an individual or population will, as a result of the 
decision, be subject to a specific City program or activity; or 
o Implicates judicial, administrative, or other forms of redress. 

• Programs that solely perform calculations for purposes of administrative 
convenience, such as adding numbers rather than performing manual addition, or 
that simply organize information, such as spreadsheets, shall not be considered a 
"computerized implementation" for purposes of ADS Task Force work. 


If any of the above questions is answered "no", then the ADS under review is n^ considered 
"Agency ADS" as defined by Local Law 49 (2018). 


^ Refer to clarification on meaning of "Decisions" provided in Question #1, at (l)(d). 
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